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We consider the problem of bandwidth selection by cross-validation from 
a sequential point of view in a nonparametric regression model. Having in 
mind that in applications one often aims at estimation, prediction and change 
detection simultaneously, we investigate that approach for sequential kernel 
smoothers in order to base these tasks on a single statistic. We provide uniform 
weak laws of large numbers and weak consistency results for the cross- validated 
bandwidth. Extensions to weakly dependent error terms are discussed as well. 
The errors may be a-mixing or Z/2-near epoch dependent, which guarantees 
that the uniform convergence of the cross validation sum and the consistency 
of the cross- validated bandwidth hold true for a large class of time series. The 
method is illustrated by analyzing photovoltaic data. 

AMS 2000 subject CLASSIFICATIONS: Primary 60F25; secondary 60G10,62G08,62L12. 
Keywords: Change-point, dependent processes, energy, limit theorems, 
mixing, nonparametric smoothing, photovoltaics. 

1. INTRODUCTION 

The nonparametric regression model, often estimated by estimators of the Nadaraya- Watson 
type, forms an attractive framework for diverse areas such as engineering, econometrics, en- 
vironmetrics, social sciences and biometrics. The present paper is devoted to a detailed study 
of a sequential bandwidth selector for kernel-weighted sequential smoothers related to the 
Nadaraya- Watson estimator. However, there are some subtle differences compared to the 
treatment of that estimator in nonparametric regression, since our Nadaraya- Watson type 
statistic is a prediction statistic, which we use to detect a change in the mean of the ob- 
servations. Addressing the detection problem, we consider a setup which differs from that 
used in classic nonparametric regression; especially, our setup leads to bandwidth choices not 
approaching 0, as the sample size increases. Thus, although there is an interesting and obvi- 
ous link to the classic regression problem, which we shall discuss in the next paragraph, the 
asymptotic results as well as the bandwidth selection problem are different and new. 

Let us assume that observations Y n = Yr n , 1 < n < T, arrive sequentially until the 
maximum sample size T is reached and satisfy the model equation 

(1.1) Y n = m(x n ) + e n , n = l,2,. ...T, T > 1, 
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for the fixed design 

x n = x Tn = G~ l (n/T), 1 < n < T, 

induced by some design distribution function G and some function m : [0, oo) — > K (as- 
sumptions on m will be given below). Tentatively, we make the assumption that the errors 
{e n : n G N} form a sequence of i.i.d.(-F) random variables such that E(e n ) = 0. We shall 
provide general results for weakly dependent time series, namely for strong mixing as well 
as near epoch dependent (NED) processes, but intend to postpone this issue to the end of 
Section 4, in order to focus on the idea of sequential cross-validation first. 

Notice that in many applications the design points are either given or selected according to 
some external optimality criterion such that it is not restrictive to assume that G is known. 
For instance, in econometrics the time instants where prices are quoted are usually fixed and 
known. Similarly, when discretizing signals or logging internet traffic, the variables of interest 
are sampled at known time points. In other applications, it may be preferable to use more 
design points in regions where m is expected to be more volatile than in other regions, or in 
regions where higher accuracy is required. The latter issue may matter, for instance, when 
analyzing the nonlinear relationship between a medical response variable and an explanatory 
variable such as age or blood pressure, or, in social sciences, e.g. when studying the influence 
of the duration of unemployment on variables measuring quantities such as political opinion 
or social networking. Hence, we can and shall assume that x n = n/T, otherwise replace m by 
m = m o G _1 , and interpret the regressor as a time variable. 

Clearly, m(t) models the process mean (signal) of the underlying observations. In practice, 
an analysis has often to solve three problems, (i) Estimation of the current process mean, 
(ii) One-step prediction of the process mean, (iii) Signaling when there is evidence that the 
process mean differs from an assumed (null) model. Usually, different statistics are used for 
those problems. For nonparametric estimation various methods have been studied including 
kernel estimators, local polynomials, smoothing splines and wavelets; we refer to Donoho &; 
Johnstone (1994), Eubank (1988), Hardle (1991) and Wand & Jones (1995), amongst others. 
Concerning procedures proposed in the literature to detect changes, there are various kinds 
of proposals. Some rely on closely related versions of those estimators, e.g. Wu & Chu (1993), 
Muller k, Stadtmiiller (1999), Steland (2005a) or Steland (20106), whereas other proposals 
construct special methods as in Pawlak et al. (2004), Rafajlowicz et al. (2008), Rafajlowicz 
&; Steland (2009) and Pawlak et al. (2010), or apply classic control chart statistics of the 
CUSUM, MOSUM or EWMA type. For the latter approach we refer to Horvath & Kokoszka 
(2002) and Brodsky &; Darkhovsky (2000), amongst many others. Frequently, change-point 
asymptotics can be based on the classic invariance principle of Donsker and its various gen- 
eralizations to dependent time series, and functional asymptotics, which plays an important 
role in functional data analysis as well; we refer to Steland (2010a), Rafajlowicz & Steland 
(2010), Bosq (1998) and Horvath et al. (2010). There is also a rich literature on the estimation 
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of regression functions that are smooth except some discontinuity (change-) points. See, for 
example, the recent work of Gijbels & Goderniaux (2004) or Antoch et al. (2007). 

Separating change detection from estimation and prediction has benefits and drawbacks. 
Of course, it allows us to apply a detector which has certain optimality properties, but this 
requires knowledge of the model after the change, which is often too restrictive for practical 
applications. Further, sequentially analyzing two or even more sequences of statistics may 
be prohibitive in real world applications. Thus, to ease interpretation and applicability, the 
present paper investigates the idea to base a detector on a prediction statistic which can be 
used as an estimator as well. Our reasoning is that a method which fits the data well and 
has convincing prediction properties should also possess reasonable detection properties for a 
large class of alternatives models. 

The proposed kernel smoother requires to select a bandwidth parameter which controls the 
degree of smoothing. As well known, the bandwidth choice is crucial for performance. The 
topic has been extensively studied for the classic problem of nonparametric regression where 
the data gets dense as the sample size increases. Cross-validation belongs to the solutions 
which have been widely adopted by practitioners. To the best of our knowledge, sequential 
cross-validation as treated in the present paper has not yet been studied in the literature. 
We propose to select the bandwidth sequentially by minimizing a sequential version of the 
cross-validation criterion. In this way, one can update the bandwidth when new data arrive. 
Since we have in mind the detection of changes where consistent estimation is not really 
required, we base our analysis on a framework which is quite common in time series analysis 
and engineering signal analysis, but differs from the nonparametric regression setting used to 
obtain consistency results. This is motivated by the fact that in many applications the data 
are observed at a scale which does not converge to 0, as the number of available observations 
approaches infinity. 

The present paper aims at presenting first results on sequential cross-validation focusing 
on uniform consistency. We establish weak and L^- consistency of the proposed sequential 
cross-validation criterion, uniformly over the time points where cross-validation is extended. 
Our results allow us to choose the number N of time points as a function of the maximum 
sample size T as well as to select their locations depending on T, as long as N grows not too 
fast compared to T. We also extend the results to obtain weak consistency uniformly over 
compact sets for the bandwidth (parameter). The results yield a consistency result for the 
optimal bandwidth under quite general conditions on the above model. Finally, we extend 
consistency to a-mixing time series and near epoch dependent series. 

The plan of the paper is as follows. Section 2 discusses our assumptions and introduces in 
detail the sequential kernel smoother of interest. In Section 3, we introduce the sequential 
cross-validation approach. Our asymptotic results for i.i.d. errors are provided and discussed 
in Section 4. Section 5 elaborates on extensions to dependent data. Those extensions work 
under very general assumptions, thus ensuring that the proposed method is valid for many 
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real data sets. Detailed proofs of the main results are postponed to an appendix. Section 6 
discusses an application of the proposal to a case study from photovoltaic engineering dealing 
with power output measurements of photovoltaics modules (solar cells). 

2. ASSUMPTIONS AND SEQUENTIAL SMOOTHERS 

Our mathematical framework is as follows. Since the information about the problem of 
interest is often not sufficient to setup a (semi-) parametric model for the process mean m 
and the distribution of the error terms, which would allow us to use methods based on, e.g., 
likelihood ratios, a nonparametric framework is employed. We assume that model (1.1) holds 
true for a function m with 

(2.1) m £ Lip([0, oo); M), either m > or m < 0, and ||w||oo < 00 > 

where Lip(j4, B), A,B C M, denotes the class of Lipschitz continuous functions A — > B. 
Clearly, cross-validation is meaningless if m = 0. Having in mind possible applications where 
one aims at detecting quickly that the process level m gets either too large or too small, we 
assume that either m > or m < and, w.l.o.g., confine ourselves to the case m > in what 
follows. Recalling that extensions to weakly dependent processes will be given in Section 5, 
let us assume at this point that {e n } are mean zero i.i.d. with common distribution function 
F satisfying 

(2.2) J x 4 dF(x) < oo. 

Under the general condition (2.1), one should use statistical methods which avoid (semi-) 
parametric specifications of the shape of m. Instead, nonparametric smoothers fh n which 
estimate some monotone functional of the process mean and which are sensitive with respect 
to changes of the mean are of interest. 

Thus, given a kernel function K : [0, oo) — > [0, oo) and a bandwidth h > the following 
sequential kernel smoother 

1 n 

rhi = fh i>h = - K([j - i]/h)Yi, i = 1, 2, . . . 

and the associated normed version 

/ 1 - 

fhi = m i:h = m h I - i]/h), 

respectively, which are closely related to the classic Nadaraya- Watson estimator, are the 
starting points of our discussion. 

Remark 2.1 At this point, it is worth noting that various classic control chart statistics 
are obtained as special cases. Denoting the target value by //q, the CUSUM chart is based on 
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Ci = ^2j = i[Xj — ([Aq + L)] where {X n } denotes the observed process and L is the reference 
value. This chart corresponds to the choice K{z) = 1, z G M, and Yj = Xj — (p>o + L) 
for all j. The EWMA recursion, fhi = XYi + (1 — A)mj_i with starting value fho = Yq, 
A G (0, 1) a smoothing parameter, corresponds to the kernel K(z) = e~' z ' and the bandwidth 
h = l/log(l — A). By defining the weights by means of a kernel function, we get a rich class 
of statistics covering classic control statistics as special cases. 

The canonical one-sided detectors (stopping times) studied in Schmid & Steland (2000), 
Steland (2004) and Steland (2005 a) have the form 

= inf{ [s T\ <i<T :fhi> c} and S+ = inf{ [s T\ <i<T : ^ < c}, 

respectively. Here c is a threshold (control limit), so G (0, 1) determines through |_^ s oJ the 
start of monitoring, and [^J denotes the integer part (floor function) of x. Notice that 
and are indeed stopping times, i.e., for instance, {S^ < n} G <r(Y"i, . . . , Y n ) for all n G N. 
A related stopping time is used in our illustration, cf. Section 6. 
Our assumptions on the smoothing kernel are as follows. 

(2.3) K G Lip([0,oo); [0,oo)), H^Hoo < oo, supp(iT) C [0,1], and K > on (0,1). 

These assumptions are quite standard and satisfied by many kernels used in practice. Our 
result on the uniform weak law of large numbers for dependent time series even works under a 
weaker condition discussed there. It is well known that the choice of the bandwidth is of more 
concern than the choice of the kernel. However, in Steland (2005a) the problem of optimal 
kernel choice for detectors based on kernel-weighted averages has been studied in greater 
detail. In this work it is shown that the optimal kernel which minimizes the asymptotic normed 
delay depends on the alternative, i.e. on the mean of the process after the change. Particularly, 
CUSUM type procedures are not optimal in general models. Although the detection statistic 
studied there slightly differs from the prediction statistic studied in the present paper, those 
results may be used to some extent in order to select a kernel, if there is some a priori 
knowledge on possible models for the mean after the change. However, in what follows we 
assume that a kernel satisfying Assumption (2.3) has been selected, such that it remains to 
select a bandwidth. 
For the bandwidth h > we assume that 

(2.4) \T/h-C\ = 0(l/T) 

for some constant £ G [l,oo), where the 0(l/T)-requirement (instead of o(l)) rules out 
artificial choices such as h = T/(£ + T~ 7 ), 7 > 0, leading to arbitrary slow convergence. 
It is worth discussing that assumption, which is rather different than the h — > such that 
nh —> 00 assumption encountered in nonparametric regression. In our setup, we work with an 
equidistant design where the distance between the time points does not converge to 0, i.e., we 
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use no in-fill asymptotics. Assumption (2.4) now guarantees that the number of observations 
on which frix depends converges to oo, as T — > oo. In practice, one can select £ and put 
h = T/£. Notice that in our asymptotic setup the parameter £ determines the degree of 
localization of the procedure. If one uses an (approximation to the) uniform kernel, £ fixes the 
percentage of observations used in each step of the detection procedure. 

The asymptotic distribution theory for procedures based on the sequential kernel smoother 
fh n has been studied in Steland (2004), Steland (2005a) and Steland (20056). Those results 
allow us to construct classic statistical hypothesis tests as well as monitoring procedures to 
detect changes in the process mean, such that certain statistical properties are asymptotically 
satisfied. Specifically, it is shown that for a large class of weakly dependent error processes 
{e^} the process {yTfh^g]^ : s S [0,1]} satisfies a functional central limit theorem with 
Gaussian limit when the underlying observations have mean 0, i.e., 




as T — > oo, for some centered Gaussian process {M(s) : s 6 [0, 1]} which depends on £; that 
result covers the limiting distribution of the classic Nadaraya- Watson estimator in our setting 
as a special case. However, it turns out that the asymptotic law and therefore the control 
limit ensuring that the asymptotic type I error rate satisfies limT_ >00 P(S^, < T) = a and 
limT'_ s . 00 P{Sj, < T) = a, respectively, depends on £, where a S (0,1). The question arises, 
how one can or should select the bandwidth h ~ T and the parameter £, respectively. 

3. FUNCTIONAL SEQUENTIAL CROSS-VALIDATION 

In the present paper, we propose to select the bandwidth h > such that the Yj are 
well approximated by sequential predictions calculated from past data Yj., . . . , Yt-\. For that 
purpose, we propose a sequential version of the cross-validation criterion based on sequential 
leave-one-out estimates. 

The idea of cross-validation is to choose parameters such that the corresponding estimates 
provide a good fit on average. To achieve this goal, one may consider the average squared 
distance between observations, Yj, and predictions as an approximation of the integrated 
squared distance. To avoid over-fitting and interpolation, the prediction of Yj is determined 
using the reduced sample where Yj is omitted. Aiming at selecting the bandwidth h to obtain 
a good fit when using sequential prediction estimates, we consider 

1 

(3.1) fh h ,-i = N^_ ^- *]/>»)*;, i = 2,3,... 

3=1 

with the constant Nt,-% = h^ 1 J2j=i ~ *]/^)- ™h,-i is defined accordingly. Notice that 
these statistics are a(Y\_, . . . , Yj_ ^-measurable, i.e. adapted. 

The statistic fhh-i can be regarded as a sequential leave-one-out estimate. In (3.1) we 
define the kernel weights using the bandwidth h; the kernel K puts a weight on the distance 
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between j/h and i/h. Due to assumption (2.4), this is asymptotically the same as putting a 
weight on the distance between the time point i/T at which we want to predict the response 
and the time point j/T. The corresponding detectors are given by 

= inf{ [soT\ < i < T : fhi > c} and 5^ = inf { [sqT\ < i <T : mi < c}, 

respectively. Given the predictions _j we may define the sequential leave-one-out cross- 
validation criterion 

CV s {h) = CV T , s (h) = fY.( Yi - ™h,-i) 2 , s e [so, 1], h > 0. 

i=2 

The cross-validation bandwidth at time s is now obtained by minimizing CV s (h) for fixed s. 
To be precise, we are interested in the following optimization problem where one minimizes 
over a set of arrays. Let Ti So ,(, be the family of all arrays {Htu '■ [soT\ < n < T, T > 1} with 

T 

lim = £ for some £ > 0. 

T->oo rtTn 

Now one considers minimizers {h^ n } € % So ,£ of the cross-validation criterion such that 
C^/r(^Tn) < C^/r(^) ; [s T\ < n < T, T > 1, 

for all {/im} G ^so,C- That procedure yields the cross- validated bandwidth /i^ \ Ts \/ T for fixed 
s. Therefore, 

hx(s) = h^ Tsi/T , s e [s , 1], 

is our functional sequential estimate for the bandwidth. 

The idea to proceed is now as follows. We shall show that CVr )S (h) converges to some 
function CV^(s) which depends on £ = lim T/h provided that limit exists. Now we expect that 
under certain conditions T/h?p, Ts ,, T converges to a minimizer of the function £ i-> CV^(s). 
That minimizer yields the asymptotically optimal constant of proportionality £*. 

Remark 3.1 The following remarks are in order. 

(i) Notice that CV s (h) is a sequential unweighted version of the criterion studied by Hdrdle 
& Marron (1985) in the classic regression function estimation framework. We do not 
consider a weighted CV sum, since we have in mind that the selected bandwidth is used 
to obtain a good fit for past and current observations. However, similar results as those 
presented here can be obtained for a weighted criterion such as T -1 X^l=i -^([L^ S J — 
i]/h)(Yi — rhh-i) 2 as well. 

(ii) At first glance, our approach is similar to one-sided cross-validation proposed by Hart & 
Yi (1998) for bandwidth selection of nonparametric regression estimators in the classic 
regression framework. However, we are interested in sequential bandwidth selection and 
aim at studying the random function s i— >■ argmin^ CV s (h). 
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Let us close this section with a discussion how to implement the approach in practice. 
Cross-validation is expensive in terms of computational costs and minimizing Ct,s for all 
s E {n/T : [sqT\ < n < T} is not feasible in many cases. Therefore and to simplify exposition, 
let us fix a finite number of time points s\, . . . , sn such that 

< so < s i < • • • < S N < 1, 

JVgN. However, for small T a small value for iV is appropriate, whereas one would prefer a 
larger value for N when T is large. Thus, it would be nice if N could dependent on T. Indeed, 
we shall later relax this assumption and allow that N is an increasing function of T. At time 
Si the cross-validation criterion is minimized to select the bandwidth, h* = h*(Yi, . . . ,Y Si ), 
and that bandwidth is used during the time interval [si, Si+i), i = 1, . . . , N. 

4. ASYMPTOTIC RESULTS FOR I.I.D. ERRORS 

The present section is devoted to a careful discussion of the asymptotic results of the 
present paper for i.i.d. errors. We provide several theorems on weak uniform consistency of 
the sequential cross-validation approach including results which allow that the number of time 
points where the cross-validated bandwidth is computed gets larger as T increases. Further, 
we show that the optimal bandwidth behaves nicely in the limit in the sense described in the 
previous section by establishing an argmin consistency result which identifies the asymptotic 
constant of proportionality under certain regularity conditions. 

4.1. Uniform convergence 

Notice that due to 

1 lTs\ [Ts\ [Ts\ 

CV s (h) = f Y, Y ?-fIl Y ^,-i + 

i=l i=2 i=2 

minimizing CV s {h) is equivalent to minimizing 

2 VTs\ 1 \Ts\ 

i=2 i=2 

Thus, we will study Cr, s (/i) in the sequel. 

Our first result identifies the limit in mean of CT, s (h) from which the asymptotically optimal 
constant of proportionality can be eventually calculated. 

Theorem 4.1 Assume (2.1) and (2.3). Then 

, ,, , , f,f L £K(£(r — u))rn(£u) dudr 

(4.i) E(c T , s (h)) -> c e ( s = -2 j ° )} . y — 

Jo (* - r )) dr 
Jo j 2 Jo Jo K (£{r ~ u))K(£(r - v))m(u)m(v) du dv dr 
j°iK{i{s-r))dr 

as T —> oo, uniformly in s G [sq, 1]. 
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It is worth mentioning that point-wise convergence holds true under weaker conditions, 
e.g., if K is bounded and continuous and m is continuous with J Q 1 m 2 (t)(it < oo. Further, 
Theorem 4.1 does not require independence as long as {e n } are pair- wise uncorrelated. For 
an example illustrating the function C^(s) we refer to our preliminary study Stcland (20106). 

We will now study the (uniform) mean squared convergence of the random function Ct s {h). 
Define Sn = {si ■ 1 < i < N}. ^-consistency holds true at the usual rate. 

Theorem 4.2 Assume (2.1), (2.2) and (2.3). Then, for any fixed integer N, we have the 
law of large numbers in Li, 

E max \C TjS (h) - E(C T>s (h))\ 2 = O^ 1 ), 
as T — )• oo. 

The question arises whether we may increase the number of time points where cross- 
validation is conducted, if the maximum sample size T increases. The following theorem 
provides such a uniform law of large numbers, but we no longer have a convergence rate. 

Theorem 4.3 Assume N = Nt is an increasing function of T and 

(4.2) < s < sjvi < • • • < saw < 1, N>1, 

and put = {sn% '■ 1 < i < N}. Given Assumptions (2.1), (2.2) and (2.3), we have the 
uniform law of large numbers in L2, 

E sup \Cr, a (h) - E(C T , s (h))\ 2 = o(l), 
as T — >■ 00, provided 




Remark 4.1 It is worth mentioning that the location of the Nt time points may depend on 
N, as long as they remain deterministic. If they are selected at random, the results remain 
valid a.s., as long as Sn and {et} are independent, since then one can condition on Sn- 

Combining the above statements, we obtain the following result. 

Theorem 4.4 Suppose Assumptions (2.1), (2.2) and (2.3) hold true and, additionally, (4.2) 
is satisfied. Then 

E sup \GrM ~ C s (0)\ 2 -> 0, 
as T —> 00, provided Nt/T = o(l). 
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We shall now extend the above results to study weak consistency of the sequential cross- 
validation bandwidth under fairly general and weak assumptions. Having in mind the fact 
that h ~ T, let us simplify the setting by strengthening that assumption to 

(4.3) h = h(0 = T/£, £€[1,3], 

for some fixed 3 € (1, oo). This means, h and £ are now equivalent parameters for each T. In 
what follows, we optimize over a compact interval, which is not restrictive for applications. 
Now rhh-i can be written as 

With some abuse of notation, let us also write 

c t 40 = CtAt/0, 

i.e. from now on the expression Ct s (T/£) is studied as a function of £ € 3. 
The optimal cross-validated bandwidth is now given by h^(s) = T/£^(s), where 

Ct{s) = argmin 5g:5: Ct, s (£)> 

if Ct, s has a unique minimum; otherwise one selects a minimizer from the set argmin^ gS Ct, s (£) ■ 
The next theorem yields weak consistency of the sequential cross-validation objective, uni- 
formly over compact sets for the parameter £ as well as uniformly over s € 5tv, where again 
Sn is the set of time points (4.2) where cross-validation is performed. 

Theorem 4.5 Assume (2.1), (2.2), (2.3) and (4.2) such that N T /T = o(l). Then, provided 
the bandwidth satisfies (4-3), we have 

(4.4) sup sup \C T ,s{0 ~ EC T , s (0\ =op(1), 
ses N 5e[l,H] 

and 

(4.5) sup sup \CtM) ~Ce(s)\ =o P (l), 
ses N es[l,H] 

as T — > oo. 

Remark 4.2 Notice that Theorem 4-5 implies weak consistency of many other Junctionals, 
for example weighted loss Junctionals 



s 



L(C T , s (0-C^s))w(s,OdsdC 



where L is a Lipschitz continuous function attaining nonnegative values, S C [so, 1] is a 
measurable set and w(s, £) is an integrable weighing function. 



SEQUENTIAL BANDWIDTH SELECTION 



11 



4.2. Consistency of the cross-validated bandwidth 



We are now in a position to formulate the following result on the asymptotic behavior 
of the cross-validated sequential bandwidth selector. The results of the previous subsection 
assuming i.i.d. errors are strong enough to apply known techniques to establish the consistency 
of minimizers of a sequence of random functions. 

Theorem 4.6 Suppose (2.1), (2.2), (2.3), (4.2) such that N T /T = o(l) and (4.3) hold true. 
Further, assume that one of the following conditions is satisfied, 
(i) C|(s) possesses a well-separated minimum = G [1,S], i.e., 



for every e > 0, or 

(ii) Ct, s {£,) is differentiable w.r.t. £ such that £ i— > ^ continuous and has exactly 

one zero. 

Then 

&{s) = argmin 5e[liS] C T , S (0 -> C> 
as T — > oo. 

Theorem 4.6 asserts that the cross-validated bandwidth h* computed at time tj = s{T, 
Si G (so, 1), is approximately given by £*.T for large T. Notice that for given K and m the 
constant of proportionality, £*., can be calculated using the explicit formulas of Theorem 4.1. 

Remark 4.3 There exist various sufficient criteria for consistency of argmin/argmax esti- 
mators, cf. Van der Vaart (1998). Condition (i) is perhaps quite suited to the present problem, 
since our assumptions already ensure uniform convergence of the sequential cross-validation 
criterion and the requirement of a well- separated minimum can be checked analytically or nu- 
merically in an application for given (K, £) and hypothesized m. Condition (i) ensures that the 
minimum is unique and especially rules out plateaus. Condition (ii) is a sufficient criterion, 
which is sometimes easier to verify, but requires the function £ i— > C^(s) to be differentiable 
with a continuous derivative. 



Many series to which detection procedures are applied are dependent time series. This 
applies to almost all data sets arising in econometrics, environmetrics and communication 
engineering, but also to many data collected in biometrics and social sciences, e.g. longitudinal 
data in clinical trials or social surveys. In this case, procedures assuming i.i.d. error terms 
are not guaranteed to be valid. It is therefore quite natural to ask whether the results of the 
previous section carry over to the dependent case. Since often the specification of a parametric 
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time series models for the error terms {et} is subject of scientific discussion, we prefer to work 
with qualitative assumptions. 

Recall the definition of the a-mixing coefficient introduced by Rosenblatt (1956). Let {Zt} 
be a weakly stationary process in discrete time. Let T 1 = a{Z{ : i < t) and Ft = a(Zi : i>t). 
Then 

a(k)= sup \P(AC\B)- P(A)P{B)\. 

The series {Zt} is called a-mixing if lim^oo a(k) = 0. For more information on mixing 
conditions we refer to Bosq (1998). Many commonly used (parametric) time series models are 
a-mixing. For instance, Carrasco & Chen (2002) establish this property for ARCH models 
under certain conditions. 

If {Zt} is a-mixing, series of the form g(Zt- m , . . . , Z t +i), m, I £ No, g a measurable function, 
inherit that property. However, infinite functions of a-mixing processes are not necessarily 
mixing. As shown by Andrews (1984), an AR(1) process with i.i.d. Bernoulli errors provides 
a well known counter-example. A more general notion is the following, which covers such 
processes. 

Definition 5.1 {Zt} is L r -NED on r > 0, if there exist nonnegative constants {dt : 
t > 1} with d t < 2\\X t - E(X t )\\r and {n : I > 0} such that 

\\Z t -E{Z t \Tl+l)\\ r <dtv h 

and V[ I 0, as I — )• oo, where T\ = cr(£j : s < i < t). 

Compared to a-mixing, near epoch dependence can be viewed as a bridge to parametrically 
motivated models such as ARMA models or, more generally, linear processes. In our further 
discussion, we shall focus on L2-NED. A L2-NED series has the property that, by definition, 
one can approximate Z t by its optimal ^-predictor H(£ t -i, ■ ■ ■ ,£,t+i) = E(Zt\£t-l> ■ ■ ■ >€t+l) 
w.r.t. the L2-norm, i.e. for any e > one can select I such that the L2-error 

u 2 = \\Z t -H{^ u ...^ t+l )\\ 2 

does not exceed e. Parametrically motivated models are usually based on some i.i.d. noise 
process Let us suppose that for some function g defined on M°° 

Zt = §{■■ & 5 6+1, •••), ieZ, 

then 

H{£t-h---,£t+i) = / •" / 9(---,z-i-i,£t-i,---,£t+i,zi+i,...)d Y[ F ( z j)> 

Zj :\j\>l 
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where F denotes the common d.f of the £ts. If, in addition, g(z) = OiZi, zi G R for i E Z, 
is a linear function with coefficients 0» S K satisfying £V < oo, i.e. 

i 

is a linear process, then we obtain 

t+l 

#(&_,,...,&+,) = ^ ^-i, t€Z. 
i=t— i 

The L2-error, given by v-i = 2(^| i | >/ |#i|)||£i||2, converges to 0, if I — >■ oo, since the coefficients 
from a l\ sequence. Thus, such linear processes are L2-NED. 

The following theorem provides the law of large numbers and weak consistency under strong 
mixing as well as a under a near epoch dependence condition. 

Theorem 5.1 Let K be a bounded kernel and suppose that 

(i) {et} is a weakly stationary a-mixing series with mixing coefficients a(k), such that 
limfe-^oo ka(k) = 0, T^Yjjj' \ Cov(ej,€ji)\ < 00 and T~ 2 ^2j t kj',k' \ Cov(eje k ,ej>e k ')\ < 
00 hold true, or 

(H) {e*} is weakly stationary with E\ei\ r < 00 for some r > 2 and L2-NED on a weakly 
stationary a-mixing process such that the above conditions hold true for that underlying 
a-mixing process. 

For fixed N € N put Sm = {s\, ■ ■ ■ , sat} for given points < sq < s\ < ■ ■ ■ < sn . Then the 
uniform weak law of large numbers holds true, 

max sup \Ct s — C^(s)\ = op(l), 

as T — >■ 00. Further, if C^(s) satisfies condition (i) or (ii) of Theorem 4-6, then for fixed 
s e [s , l] 

p 

argmin €g[1H] C T , S (0 -> C s , 
as T —> 00. 

Notice that the results work under a less restrictive moment assumption and also allow for 
a larger class of kernels. Indeed, the kernel may have an unbounded support and is allowed 
to take negative values. However, we have to assume that the number TV" of time points at 
which cross-validation is conducted is fixed. 

6. ILLUSTRATION: AN APPLICATION IN PHOTOVOLTAICS 

To illustrate the approach, we report about the following simulation experiment where we 
applied the method to a photovoltaic problem using real data to simulate error terms. In 
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TABLE I 

Simulation estimates of the mean delay for some values of A. 



A 


2/3 


2/3 


4/3 2 4 


Mean delay 


60.25 


37.39 


10.01 4.12 2.07 



photovoltaics the power output of photovoltaic modules is the most important quantity for 
quality assessment, cf. Steland Sz Herrmann (2010). In a scenario analysis we simulated a 
series of measurements according to the change-point model 



Y t = fi(t;e) + e t , t = l, 



where 



mM) 



Mo, 1 < t < qi, 

fjLo + (t- qi)6i, qi<t< q 2 , 



Mo + (?2 - gi)<5i + A, q 2 < t, 

for t = 1, . . . ,T with 8 = (<5i, 8 2 , qi, (72)' and T = 386. Mq = 200 denotes the nominal (target) 
power output, the parameters 5\ = —0.1 and A model a drifting decreasing quality in terms of 
the mean power output with breaks (change-points) at q\ = \T/4:\ and q 2 = \ T/2\. If A = 0, 
then the process stabilizes after q 2 having a constant mean of 180.7; for this case study 
modules with power output larger than 180 were regarded as acceptable after re-labelling. 
Otherwise, there is a level shift of size A; for the scenario analysis we put A = 2s ~ 4.3 where 
s denotes the standard deviation of the errors. The errors were drawn from a real data set of 
photovoltaic measurements to mimic a real-world setting in the simulation. A = represents 
the null model (null hypothesis Hq) of interest for our analysis. 



The simulated data and the cross- validated sequential kernel smooth are depicted in Figure 1 
and accompanied by a control limit c. It can be seen that the predictions rhh-i provide a 
reasonable approximation to the process mean. We applied the detector using a Gaussian 
kernel, i.e., K(z) = (27T)- 1 / 2 exp(-z 2 /2), z G R, and the cross-validated bandwidth where 
cross-validation was conducted at the time points 50, 100, . . . , 350. The start of monitoring 
was determined using the rule min(25, h^(si)) and the control limit was obtained by Monte 
Carlo simulation ensuring an in-control average run length of ARLq(S^) = Eq(S^) ~ 350 
yielding c ~ 178.79. The signal is given at time instant 296. Table I provides simulation 
estimates of the mean delays defined for our purposes by Emax(0, S^, — q 2 ), as a function of 
A. Again, the errors were simulated from real measurements. It can be seen that the chart 
reacts quickly to jumps. 
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Figure 1. — A scenario analysis for photovoltaic measurements. True model (thin line), 
sequential smooth (bold) and control limit (red horizontal line). 
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APPENDIX A: PROOFS 

Proof: (of Theorem 4.1). 

I rji a I 

Since the norming function s H> N T (s) = h' 1 ^XJ J K([[Ts\ -i]/h), s £ [0, 1], is deterministic 
and converges to N^(s) = £ K(£(s—r)) dr, we may and will assume that Nt(s) = Ng(s) = 1. 
First note that for j < i we have E(YiYj) = m(i/T)m(j /T), since E(ej) = E(ei) = and 
E{ei€j) = by independence. Further, E(Y?) = m(j /T) 2 + E{e 2 ). We have the decomposition 
E{C T , s {h)) = J$], + J [ tI + 4 3 1 where 

2 LTsJ 1 1-1 

j t? s = - f E i E - mmYj), 

i=2 j=l 
, ITs} f .2 i-1 

J ff = ^EU S K([i-j]/h)m-k]/h)E(Y 3 Y k ), 

i=2 ^ ' j,k=l,j^k 

4 3 i = ^E(T) E^-^W)- 

i=2 V 7 3=1 
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2^ 

We provide the arguments for the second more involved term JL'fh), the other terms are 
treated similarly. Notice that 

, ITs} . .2 i-i 
J ff = t E U E " d/WP " k]/h)m(j/T)m(k/T) 

i — O \ ■/ A I 1 A-tU 



i=2 j J k=l,j=£k 
2 



ITs} r _l r _l 
T r, T T 

~T~ 

h 

2_ _i J. 

T T T 



fT/h( u > v -, r ) dudvdr 



where 



f T/h (u,v,r) = K ) m {^r) m {-^) 

Since K as well as m are Lipschitz continuous and bounded, we have 

h/h( u , v > r ) /?( n ' w ' r ) = £ 2 ^(£( r ~ «))K(£(r - u))m(u)m(t;), 

as T — )• oo, uniformly in u, u, r € [so> 1]- Notice that point-wise convergence holds true under 
weaker conditions, e.g. K bounded and continuous and m continuous with f m 2 {t) dt < oo, 
by dominated convergence. Now the result follows easily for the case Nt = 1, 

J^l Jp) = £ 2 / / / ^ (£(r - (r - v))m{u)m{v) du dv dr, 

Jo Jo Jo 

uniformly, as T — > oo. To handle the general case, consider the analogous decomposition, 
C T , s (h) = 4> + J 2> + J 2>> where for instance J^ s = J^jN^(s). Put J s {2) = JP/N^s) and 
notice that 

{ } N*(s) Nl(s) Nl(s)N*(s) 
Since K > on (0,1), 

sup Nt 2 (s), sup (/ (a - u)) du) =0(1), 

86[sq,1] s6[s ,l] \Jo J 

provided T is large enough. Further, 

sup|iVf (s) - iV| (s) | < 2sup|7V T (s) - ^(^lOdlKHooLTsJ/Zi) = o(l). 

s s 

Note that the numerator in (A.l) equals 

[A r|( s ) _ AT|( s )]j( 2 ) + - jP)Nl(s), 

which converges to 0, uniformly in s 6 [sq, 1], since ||J (2) ||oo, Halloo < oo. Q.E.D. 
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Proof: (of Theorem 4.2) 

To simplify the proof let us first assume that Nt-% = 1- Recall that for j < i we have 
E(YiYj) = m(i/T)m(j /T), since £7(ej) = i2(ej) = and E(ei€j) = by independence. Thus, 

(A.2) Cij = - ^(y^) = e im (j/T) + m(i/T)ej + 

and 

(A.3) Cii = - = e 2 " + 2 ^Mj/T). 

Notice that J5(C»j) = 0, and E(C,f-) < oo, since, e.g., E{e i e j ) A = (Eef) 2 and E(eim(j /T)(e*6j) 3 ) < 
||m||oo^(ef )£^|ei| 3 . Also note that E(Y 2 ) = m(j /T) 2 + E(e 2 ). Consider the decomposition 

(A.4) Qr, a (fc) - E(CtM) = Ut,s + V T>S + W T , S 
where 

2 LTsJ 1 

(A.5) C/ T)S = rE^['-^)fe' 

i=2 3=1 

1 LTSJ 1 % - x 

(A.6) ^ T , s = -^ p X) ^([i-i]/W[i-*]A)Cifc, 

i=2 j,k=l,j^=k 
~ 1 L ^ J 1 ^ 

(A.7) fT, s = ^pE " 3\/h)%r 

i=2 j=l 

By virtue of Loeve's C r -inequality 

(A.8) E\C T>a (h) - E(C T , s (h))\ 2 < A{E\U T ,s? + E\V T , S \ 2 ) + 2E\W T , S \ 2 . 
Let us first consider £J|Vr,<j| 2 which can be written as 

[Taj i-l i'-l 

(A- 9) ^E E E K([i-j]/h)K([i-k]/h)x 

i,i'=2 j,k=l,j^k j' ,k'=l,j'^k' 

K{\i' - j']/h)K([i' - k']/h)E(( jk ( jlk ,). 

Notice that -E(CjfcCj'fc') equals 

E(ejm(k/T) + m(j/T)e k + e j e k )(efm{k' /T) + m(j'/T)e k > + eye^) 

and vanishes, if {j, k} n {j' 3 A;'} = by independence. Since if has support [—1, 1], the sums 
^2j^ k an d Ylj'^k' concern only terms with \i — j\,\i — k\ < h and \i' — j'\, \i' — k'\ < h. Now 
consider the remaining non-vanishing cases. For the 0(h 2 ) terms with j = j 1 ^ k! = k, we 
have 

E(( jk ( f k>) = E(e im (j/T) + m(k/T)e 2 + eie 2 ) 2 , 
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which is non-negative and bounded in j, k, T. For the 0(/i 3 ) terms where j = j' and k ^ k' 
notice that j = j' ^ k' . Therefore, 

E{e j m{k/T)m{3 , lT)e kl ) j t= k ' 0, E{e j m{k/T)e jl e kl ) j tt' 0, 

E(m(j/T)e k e f m(k'/T)) h t k ' 0, E(m(j/T)e k m(f /T)e k ,) ^ 0, 

E(m(j /T)e k €j>e k i) k ^ k = ^ 0, E(eje k m(j' /T)e k <) k ^ k ' k =t j j 0, 

and E(eje k ejie k i) ^ == 5 ' 0. Thus, E{Q k Qji k r) = E(e\m(k/T)m(k' /T)) which is non-negative 
and finite. This shows that the contribution of these terms is not larger than 0(T~ 2 h~ 4 h 3 ) = 
0(T~ 3 ). We may summarize that there exists a constant c not depending on s such that (A. 9) 
is not larger than cT~ 3 . Consider now 

[Ts\ i-l i'-l 
i,i'=2 j=l j'=l 

If {i, j} n {*',;'} = 0, then E(djCi'j') = by independence. If i = i' and j / j' {0(Th 2 ) 
terms), or i ^ i' and j = j' (0(T 2 h) terms), or i = i' and j = j' (0(Th) terms), again we 
have < EfajQ/ji) < c < oo for some constant c yielding 

E\U T , S \ 2 < 4KWIT- 1 , 

where c does not depend on s 6 [0, 1]. We may conclude that the resulting upper bound for 
(A.8) is 0(T _1 ) for all s G [0, 1] yielding 

sup E\C T , s (h) - E(C TyS (h))\ 2 = 0{T- 1 ). 
se[o,i] 

The proof is now completed as follows. 

E sup \C T , S - E(C T ,s)\ 2 <EY, \Ct,s - E(C TyS )\ 2 
sgSn ses N 

< \S N \ sup E\C T , S - E(C T ,s)\ 2 
se[o,i] 

= 0(NT- 1 ) = 0{T- 1 ), 

as T — > oo. 

Let us now discuss the modifications when using m n -i instead of m n —j. Denote the cor- 
responding decomposition of CT, s {h) — E(CT, s (h)) by Ut >s + Vt,s + Wr >a , where the kernel 
weights K([j — i]/h) are replaced by K([j — i]/h)/ Ylk=l ~ (l/^)- We show how to treat 
Ut, s - Notice that E\UT, s (h)\ 2 can be represented as 

r[Ts\ r[Ts] rr-l/T rr-l/T rr'-l/T 

(A. 10) / / / WT(r)wT(r') / / GT,h(u,v,r,r') dv dudr dr' 

J2/T J2/T Jl/T Jl/T Jl/T 
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where 

< G T , h (u,v,r,r') = K([[Tu\ - [Tr\]/h)K([[Tv\ - [Tr'\ ] / h)E((^ Tr j ) |_ Tlt j C|Tr'J , |TuJ ) 

and 

/LTrj-l , 

«*(r) = 1 / J2 K([[Tr\-j]/h)^w(r) = l^ ^ K(£(r - *)) 

Here S(CLTrJ,|T«J C|Tr'J,|T?)J ) stands for the function 

(r,u,r',v) !-)• ^(CijCi'j') 1 ! 7- S G X,-,r' G G 

where 2j = [i/T, (i + 1)/T) for < i < T. Since 7 = inf s6 r S0)1 i f£ K(£(s — z)) dz > 0, the 
elementary fact |l/x n — l/x\ = |(x — x ra )/(xx n )| yields sup r \wt{t) — w(r)\ = 0(1/T). Thus, 
if we replace in (A. 10) the functions wj<(r) and wt{t') by their limits, by nonnegativity the 
difference can be bounded by 

ATs\ ATs\ rr-l/T rr-l/T rr'-l/T 

sup \wT(r)wT(r')— w(r)w(r')\ / / / / / GT t h(u,v,r,r') dv dudr dr' 

r J2/T J2/T Jl/T Jl/T Jl/T 

which is of the order 0(1/T). To estimate the remaining term, namely 

[Ts\ i-[Ts\ ,-r-l/T pr-l/T pr'-l/T 

/ / w(r)w(r') / / GT,h{u,v,r,r') dv dudr dr' , 

2/T J2/T Jl/T Jl/T Jl/T 

notice that sup r w(r) < I/7 < 00. Thus, the expression in the last display is not larger than 

/•[TsJ flTs] ,-r-l/T ,-r-l/T pr'-l/T 

7~ 2 / / / / / Gx,h(u,v,r,r') dv dudr dr' , 



'2/T J2/T Jl/T Jl/T Jl/T 

which equals j- 2 \EU TtS (h)\ 2 . Q.E.D. 
Proof: (of Theorem 4.3) 

By virtue of the method of proof used to establish Theorem 4.2, the result follows at once 
from £sup s&Sjv \C T>S - E(C T , S )\ 2 = O^tT^ 1 ) = o(l). Q.E.D. 

Proof: (of Theorem 4.4) 

Define for s G 5jv X s = C T , s (h) - E(C T , s ( h )) and 3>« = E(C T , s ( h )) ~ C s(0- Due to Theo- 
rem 4.2, we have Eswp se . s \X S \ = o(l) and Theorem 4.1 yields E$up aeS \y s \ = o(l). Using 
the estimate \X a + y s \ < max{2\X B \,2\y a \), which yields \X S + y s \ 2 < 4 max(\X s \ 2 , \y s \ 2 ) < 
4:(\X S \ 2 + \y s \ 2 ), we obtain 

E sup \C T , s {h) ~ C s (0\ 2 = E sup \X s + y s \ < 4 sup \X S \ 2 + 4 sup \y s \ 2 , 
which completes the proof. Q.E.D. 
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Proof: (of Theorem 4.5) 

By Theorem 4.1 it suffices to verify (4.4). We make use of the decomposition for Ct s(h) — 
ECT, s (h) obtained above with the substitution h = T/£ and discuss the corresponding term 
Vr tS (£) i n detail. The other terms are treated analogously and omitted. Fix some £' G [1)3] 
and S > 0. For brevity of notation, we will use the notation sup^ = snp^ e r^_g^t + g\ for the 
next steps and put 

(A.ll) w T , ljk = t 2 m(i - j)/T)K{H{i - k)/T). 

The inequality sup^, \f(x)\ < | sup x . f(x)\ + | inf^ f(x)\ yields 



sup \v T , s (0\ < 2(<i(C0 + ^J(0 + v^(O) 



r(3). 



where 



1 w 1 w 

^2 ^2 m P w T,ijk(Q(jk - ^^2E(supWT,ijk{Q(jk 



Here £r) fc signifies J2i=2 Y?j,k=l,frk- Since -^1 SU P? w T,ijk(0(jk\ S < ll-^llL s IOfc| 4 < oo, one 
can verify that for fixed £' 



e( S u V V^ s (C')) =o(l) and E(saj>V$(e)\ = °0) 

\s£Sn J \seSM J 



using the same arguments as in the proofs of Theorem 4.3 and 4.4. Let us now show that for 
fixed £' the term Vft(g) can be made arbitrary small. Indeed, by Lipschitz continuity of K, 
we may choose 5 > small enough to ensure that 

max (sup - inf)£lf(£(i - j)/T)£K(£(i - k)/T) 

is arbitrary small. Thus, by boundedness of K and the dominated convergence theorem, for 
any e > the mean of each of the X]|=i *(* ~~ 1) = ®( [Ts\ 3 ) summands of Th 2 V^ 3 j (£') (recall 
the definition (A. 6)) is smaller than e, uniformly in i,j, k, if 5 > is sufficiently small. Then 



E 



sup Vgte 



sGSn 



Th 2 
(i) 



^<3,k<T £ 



£2 E £ k 111 ^^ " inf )^(^ " 3)/T)iK{i{i - k)/T)\( jk 



Th 2 

0{T 3 /{Th 2 )e) = 0(e). 
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By compactness, a finite number of open balls B(£, 5) = (£ — 5, £ + 5) cover [1, H], such that 
[1,3] C + for 6 [1,E], where M = M(e) E N depends on e. Thus, 

using the union bound we may now conclude that 

PI sup sup |Vr )S (C)| >e) < -P{ max SU P SU P |^7>(£)l > £ 



KKN 



< 



Vpfsup sup F T)S (0>e/M > ) 
" \seS N eeB(^',<5) / 



A/ 



<^p(2sup sup £ 

<^p(2su P x; 4? & 



> e/M 



We arrive at 



l=l 



(A.12) p( sup sup ^..(Ol > e) <J2 Yl P ( SU P ^ 

\^e[i,s] seSjv / j=i i/=i2,3 \se5jv 



i/=l,2.3 



> e/M 



£(eO> £ /(6M)Y 



with 



pfsup y^te') > e /(6M)) = o (e( sup 

VseSjv / y \s65jv 



^ = 1,2, 



and 



pQup vg(0 > e/(6M)j = O \E 
This completes the proof, since M is finite. 



sup V®(C, 



Q.£.Z>. 
Q.P.D. 



Proof: (of Theorem 4.6) 
Confer Van der Vaart (1998, Ch. 5.2). 

We shall make use of the following coupling lemma, cf. the works of Bradley (1983, Theo- 
rem 3), Schwarz (1980) and Bosq (1998, Lemma 1.2), which allows to approximate directly 
dependent random variables by independent ones, a technique introduced in the papers Berkes 
& Philipp (1977/78) and Berkes & Philipp (1979). 

Lemma A.l (Bradley /Schwarz lemma). 

Let (X, Y) be aR d x M-valued random vector such that Y £ L p for some p G [1, +oo]. Let c 
be a real number with \\Y + c\\ p > and £ G (0, \\Y + c\\ p ). Then there exists a random variable 
Y* such that Py* = Py , Y* and Y are independent and 

P(\Y-Y*\ >£) < ll^Y + c\\ p yl^ +l \a{X,Y)f p/{2p+l) . 



Here a(X,Y) denotes the a-mixing coefficient between cr(X) and cr(Y). 
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Notice that in Lemma A.l one may assume a(X,Y) > 0, for otherwise the assertion is 
trivially satisfied with Y* = Y, cf. Bradley (1983, p. 76). 

Proof: (of Theorem 5.1) 

For weak consistency it suffices to show that either 
snv\C T , s (0-G^s)\=o P (l), 

(and £ i— > C^(s) has a well-separated minimum), or 

C TjS (0 -> C ( (s), for each £ e [1,3] 

(and £ i— > Cr, s (£) has an unique zero for each T). Arguing along the lines of the proof of 
Theorem 4.5 we see that it suffices to show 

(A.13) V$(?)->0, k = 1,2, 

in probability. Indeed, since Sn is a finite set, the right-hand side of (A. 12) is not larger than 
the finite sum 

M N 

(A- 14) £ £ Y, P (V^ s >e/{QMN)). 

1=1 i/=l,2,3 s=l 

For i.i.d. errors with fourth moments this is a consequence of our results on uniform L2 
convergence. To extend the result to the dependent case, we apply the Bradley/Schwarz 
lemma A.l and the block-splitting technique to the random variables V^(£'), k = 1,2. We 
have to check that their second moments are bounded by some constant U < 00 (which 
depends on H^T-Hoo, ||-fif||oo, -^(^i) and ^). Let us sketch the arguments for k — 1. Recall the 
definition (A. 11) and the representation (A. 9) to see that 

[Ts\ i-i i'-i 

e ( v t]1(0) 2 < ^4 Y Y Y 1 w T ,ijkw T ,i'j'h' 1 1 Gov (Qk, Cfk' ) 1 

i,i'=2 j,k=l,j^k j',k'=l,j'^k' 
I 1 l ~ X ''^ \ 

(A.15) =oL E Y \ Cov (CjkXf k ')\ . 

\ j,k=i,j^kf,k'=i,j'^k' j 

Notice that the estimate (A.15) requires the kernel K only to be bounded. Recalling ||?n||oo < 
00 and the definition of the (jk in (A. 2) and (A. 3), respectively, we get 

\Cov((jk,(j'k')\ < max{||m||oo, l}(|Cov(ej,ej/)| H h |Cov (e 3 -e fc , ef€ k ')\). 

Estimating separately the resulting sums obtained when combining that inequality with 
(A.15), we see that E(vjQ(g)) 2 = O(l), uniformly in s, since T -1 J2jj> \Cov (ej , e f )\ < 00 
and T~ 2 ■ k j' k' I Gov (ejek, e/efc')! < 00. In other words, the second moment of Th 2 V^ s {^'), 
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a sum of 0(T 3 ) summands, is 0(T 6 ), and the same applies whenever selecting, say, p sum- 
mands and considering the second moment of their sum S p , i.e. E(S 2 ) = 0(p 2 ). 

Let Vr,s{€') e ■ k = 1,2} and denote the m = 0{Th 2 ) summands by piju such 

that i7iVr,s(£') = Si=i^ J2]~k=i j^k Pijk- Let us now apply the block-splitting technique in 
combination with the Bradley/Schwarz coupling lemma. Notice that the following derivations 
do not depend on the kernel at all. For simplicity, we shall assume m = 2pq, where p and 
q will be chosen later. Partition the m summands in consecutive blocks of length p. Clearly, 
mVr,s(0 * s the sum of the 2q partial sums of these blocks. Number these partial sums from 
1 to 2q and denote the partial sums corresponding to odd numbers by B±, . . . , B q and those 
corresponding to even numbers by B[, . . . , B' q . Let e > 0. It suffices to establish a bound for 
P{\Bj\ > me). Put c = 2pU and notice that 

min \\Bj + c 1 1 2 > c — max ||-Bj||2 = pU, 

j j 

since H-B7H2 < pU by Minkowski's inequality, for all j. Let 
£ = mm(pU, me/(4g)) 6 (0, min \\Bj + c|| 2) - 



Applying Bradley's lemma yields the existence of B* independent from £?*,... ,B*_ 1 with 



B* = Bj and 



P(\B 3 -B*\>0< 11 {^j^) 



2/5 



a. 



< 11 



3p£/ 



(P) 4/5 

\ 2 / 5 



^min^t/, me/ (4g)) 
< ll(3 + 6C//e) 2/5 a(p) 4/5 . 



J <*(p) 



4/5 



By independence of .Eq , 



,5* and since E{B*) 2 



-,2^.2 



> me/4 < 



m 2 e 2 /4 



= 0(p 2 ), we have 
0(4/(2ge 2 )). 



The inclusion 





1 

E^ 


me ] f 


E^ 


{ 













^ m£ IR 



B*\<^l<j< 



q\u(j{\B J -B*\>t}. 
J .7=1 



leads us to 
P 



E^ 



> 



me\ 



2 J 



me\ 



1 < p 

< Odqe)- 1 ) + 0( 9 (3 + 6?7/e) 2 / 5 a(p) 4 / 5 ) 
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Putting p = [m 1 / 2 ] and q = m/(2p), we may conclude that the last expression is o(l), 
provided lim^oo ka(k) = 0. This shows that (A. 13) holds for the strong mixing case, 



(A.16) P(V t [ k s \0>e)<P 



3=1 



> 



me 



o(l). 



Let us now discuss the case that {et} is L 2 -NED on a stationary a-mixing process such the the 
above arguments hold true when the error process {et} is replaced by that a-mixing process. 
As in Definition 5.1, denote the corresponding a-fields by J-*. Put ?t = E{et\J :t t ^ l l ) for all t 
and / > 1. Then [| e* — 1 1 2 = 0(y{) for some sequence z/; = o(l), as I — > 00, uniformly in t. 
Recall the definitions (A. 2) and (A. 3). Denote by Vr, s (h) the statistic Vr )S (/i) where the 
are replaced by the ?j i.e. Cij are replaced by the random variables 



E(ei7j) + e im {j/T) + ej m(i/T). 



We have 



||Cy - Cijh < \\e%ej - egjWi + \\E(ei€j - e{ej)\\i 

+ \\m(i/T)(ej -e.OHi + \\m(j/T)(€i 

< 2 1| CiCj - ?i?j||l + 2H7T7.IIOO ||ei - ?j||l 

< 2(||ei|| 2 ||ej - €j|| 2 + ||e/||2||ei - ?j|| 2 + ||m||oo||ej - ?i|| 2 ) 



0{vi) 



For fixed £' we obtain 



sup ||Vr, s (0 - ^, s (C')lli = SU P 
se[so4] se[so4] 



[Ts\ i-l 

X! X] w T,ijk((ij ~ Cij) 
i=l j,k=l,jj^k 



Oiyi) 



which allows us to estimate the summands in (A. 14) by 

P (VS > e/(6MN)) = P(V$ > e/(6MN)) + sup \\V^} - ^||i 



<o(l) + 0(yi), 



se[s 0) i] 



since V^s is calculated from the a-mixing random variables ej, such that (A.16) applies. This 



completes the proof. 



Q.E.D. 
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